Using RStudio on Hoffman2

Hoffman2 Happy Hour

Charles Peterson

Hoffman2 Happy Hours

🎉 Welcome to the Hoffman2 Happy Hours

  • Short presentations on topics related to HPC and Hoffman2
  • Thoughts for future “Happy Hour” topics? 💡

📧 cpeterson@oarc.ucla.edu

Files For This Presentation

This presentation can be found on our github page

https://github.com/ucla-oarc-hpc/H2HH_rstudio


The html slides can be found at

https://ucla-oarc-hpc.github.io/H2HH_rstudio


More information and scripts on using RStudio on Hoffman2

https://github.com/ucla-oarc-hpc/H2-RStudio

RStudio Information

What Is RStudio

RStudio is a great IDE for R and visualize files.


But why do you want to use RStudio on Hoffman2 when you can use your own computer???

RStudio on Hoffman2 provides access:

  • higher memory
  • multi-core
  • GPUs
  • Your data on Hoffman2

RStudio Formats

There are two main (free) RStudio formats that researchers can use


RStudio Desktop

  • Standalone desktop application

  • Installed locally on your machine

RStudio Server

  • Run RStudio as a server process
  • Open on a web browser

RStudio on Hoffman2

RStudio Desktop can be inefficient on Hoffman2

  • require X11 forwarding
  • slow
  • sluggish interaction

RStudio Server is the best way to use RStudio on Hoffman2

Running RStudio

Running RStudio (1)

Get An Interactive Job

  • Containers cannot run on login nodes.
    • You MUST use a compute node


qrsh -l h_data=10G

Running RStudio (2)

Create Temp Directories

  • Create writable temp directories
    • RStudio writes small files
    • Anywhere you have write access



mkdir -pv $SCRATCH/rstudiotmp/var/lib
mkdir -pv $SCRATCH/rstudiotmp/var/run
mkdir -pv $SCRATCH/rstudiotmp/tmp

Running RStudio (3)

Load the Apptainer Module

  • Apptainer is software that will run the Rstudio container


module load apptainer

RStudio Server on Hoffman2 created from Docker

  • Separate R from modules on Hoffman2
    • DO NOT load R modules
    • R packages may need to be reinstalled

Running RStudio (4)

Start Up RStudio

apptainer run \
 -B $SCRATCH/rstudiotmp/var/lib:/var/lib/rstudio-server \
 -B $SCRATCH/rstudiotmp/var/run:/var/run/rstudio-server \
 -B $SCRATCH/rstudiotmp/tmp:/tmp \
 $H2_CONTAINER_LOC/h2-rstudio_4.1.0.sif
  • apptainer run
    • Starts the RStudio container
  • -B $SCRATCH/rstudiotmp/[dir]:[/dir]
    • Mounts tmp directories to the container
  • $H2_CONTAINER_LOC/h2-rstudio_4.1.0.sif
    • Location of RStudio container
    • Can be change to different RStudio versions
  • Information will display about RStudio session
    • Note the compute node name and port number.
    • Displays ssh -N -L ... info to be ran

Note

KEEP THIS TERMINAL OPEN UNTIL YOU JOB IS DONE

Running RStudio (5)

  • Open another terminal on your local computer

  • Run the port forward command

    • Creates a connection from local computer to compute node
ssh  -L 8787:nXXX:8787 username@hoffman2.idre.ucla.edu 
  • Change port 8787 if needed
  • nXXX is the compute node name
  • username is your Hoffman2 username

Running RStudio (6)

  • Finally, open a web browser
    • Type URL of RStudio Server
    • Will ALWAYS be localhost
    • Change port 8787 if needed
http://localhost:8787

Running Rstudio - The Easy Way

  • h2_rstudio.sh
    • Script that runs everything from the previous slide
    • Starts Rstudio and opens a web browser for you
    • Runs on your local computer (not Hoffman2)
  • Download script
wget https://raw.githubusercontent.com/ucla-oarc-hpc/H2-RStudio/main/h2_rstudio.sh
chmod +x h2_rstudio.sh
  • To display how to use this script
./h2_rstudio.sh -h
  • Run script
    • Replace username with Hoffman2 username
./h2_rstudio.sh -u username

h2-studio.sh Information

Look at our Github page

Tested Platforms

Mac’s terminal app

Window’s WSL2

MoboXterm

GitBash

RStudio Script

This RStudio Script is currently on our GitHub page

Information on this RStudio Container (1)

  • Rstudio container was built using Docker
    • Based on RStudio images from the Rocker Project
    • Hoffman2 containers located at $H2_CONTAINER_LOC
    • RStudio containers are named h2-rstudio_X.Y.Z.sif
      • Where X.Y.Z is the R version
  • View all available RStudio containers by running
module load apptainer
ls $H2_CONTAINER_LOC/h2-rstudio*sif

Information on this RStudio Container (2)

  • Separate build of R and
  • R packages installed in unique directory
    • ~/R/APPTAINER/h2-rstudio_4.1.0 (for h2_rstudio-4.1.0.sif)
  • HPC Container files
    • Docker and definition files for Hoffman2 containers
    • RStudio Dockerfiles have all you need to build RStudio

R Package Installs

  • Some R packages require extra libraries or software in the container
  • Contact us to update this container
    • OR you can modify the Dockerfile for your own container

Tips for Running RStudio (1)

  • If Rstudio does not at start up
    • Possibly due to previous RStudio not shutdown correctly
  • Clear out any tmp directories config files
rm -rf ~/.config/rstudio
  • Clear out RStudio config files
rm -rf $SCRATCH/rstudiotmp

Tips for Running RStudio (2)

  • Access to a Hoffman2 terminal in RStudio

Using Batch R

  • Instead of interactive RStudio, you can run R as a non-interactive batch job
    • Use R from inside RStudio container as a qsub job
  • Create a job script
    • Load Apptainer
    • use RStudio container with a .R script
      • apptainer exec
#!/bin/bash
#$ -cwd
#$ -o rstudio_batch.out.$JOB_ID
#$ -j y
#$ -l h_rt=3:00:00,h_data=10G
#$ -pe shared 1

# Load Apptainer module
. /u/local/Modules/default/init/modules.sh
module load apptainer

# Run R with a R script, named myRtest.R
apptainer exec $H2_CONTAINER_LOC/h2-rstudio_4.1.0.sif R CMD BATCH myRtest.R
  • Then run this job script
qsub rstudio_batch.job

Summary

  • Utilize RStudio Server on Hoffman2
    • Access through on your web browser
    • Applicable to other HPC resource as well


  • RStudio can be used interactively or as a non-interactive batch job


  • Use the h2_rstudio.sh script for easy setup

Thanks and Happy Computing!

Questions? Comments?